Estimating Probabilities in PCFGs

Estimating Probabilities in PCFGs

Probabilistic Context-free Grammars in Natural Language Processing

Context-free grammars (CFGs) are a class of formal grammars that have found numerous applications in modeling computer languages. A probabilistic form of CFG, the probabilistic CFG (PCFG), has also been successfully applied to model natural languages. In this paper, we discuss the use of PCFGs in natural language modeling. We develop PCFGs as a natural extension of the CFGs and explain one prob...

Statistical Properties of Probabilistic Context-Free Grammars

We prove a number of useful results about probabilistic context-free grammars (PCFGs) and their Gibbs representations. We present a method, called the relative weighted frequency method, to assign production probabilities that impose proper PCFG distributions on finite parses. We demonstrate that these distributions have finite entropies. In addition, under the distributions, sizes of parses ha...

Parsing Inside-Out

Probabilistic Context-Free Grammars (PCFGs) and variations on them have recently become some of the most common formalisms for parsing. It is common with PCFGs to compute the inside and outside probabilities. When these probabilities are multiplied together and normalized, they produce the probability that any given non-terminal covers any piece of the input sentence. The traditional use of the...

Can Probabilities Be Mimicked by Rules?

We examine the expressive power of probabilistic context free grammars (PCFGs), with a special focus on the use of probabilities as a filtering mechanism. Probabilities in PCFGs induce an ordering relation among the set of trees yielding a given input sentence. PCFG parsers return the trees bearing the maximum probability for a given sentence, discarding all other possible trees. Obviously, thi...